This paper presents predictive gain scheduling, a technique for simplify(cid:173) ing reinforcement learning problems by decomposition. Link admission control of self-similar call traffic is used to demonstrate the technique. The control problem is decomposed into on-line prediction of near-fu(cid:173) ture call arrival rates, and precomputation of policies for Poisson call ar(cid:173) rival processes. At decision time, the predictions are used to select among the policies. Simulations show that this technique results in sig(cid:173) nificantly faster learning without any performance loss, compared to a reinforcement learning controller that does not decompose the problem.

In multi-service communications networks, such as Asynchronous Transfer Mode (ATM) networks, resource control is of crucial importance for the network operator as well as for the users. The objective is to maintain the service quality while maximizing the operator's revenue. At the call level, service quality (Grade of Service) is measured in terms of call blocking probabilities, and the key resource to be controlled is bandwidth. Network routing and call admission control (CAC) are two such resource control problems. Markov decision processes offer a framework for optimal CAC and routing [1].